Subspace-Aware Exploration for Sparse-Reward Multi-Agent Tasks

Exploration under sparse rewards is a key challenge for multi-agent reinforcement learning problems. One possible solution to this issue exploit inherent task structures an acceleration of exploration. In paper, we present novel exploration approach, which encodes special structural prior on the reward function into exploration, sparse-reward tasks. Specifically, entropic objective proposed accelerate discovery rewards. By maximizing lower bound objective, then propose algorithm with moderate computational cost, can be applied practical Under setting, show that significantly outperforms state-of-the-art algorithms in multiple-particle environment, Google Research Football and StarCraft II micromanagement To best our knowledge, some hard tasks (such as 27m_vs_30m}) have relatively larger number agents need non-trivial strategies defeat enemies, method first learn winning setting.